1. Technical Field
The invention is related to synthesizing textures for arbitrary surfaces, and in particular, to a system and method for computationally efficient synthesis of bidirectional texture functions (BTF) on arbitrary surfaces given natural or synthetic BTF samples as inputs.
2. Related Art
Generating textures on arbitrary surfaces has been an active area of research. One approach is to simply map pre-computed texture patches onto a target surface. For example, a conventional “lapped texture” technique randomly pastes texture patches onto a surface following orientation hints provided by a user. To hide mismatched features across patch boundaries, textures in the overlapping regions are blended. This technique works well for many textures, but for highly structured textures and textures with strong low-frequency components, the seams along patch boundaries are still evident.
A number of other conventional schemes have been proposed for directly synthesizing textures on arbitrary surfaces. For example, a multi-resolution synthesis algorithm has been adapted for directly synthesizing textures on arbitrary surfaces. However, schemes based on directly synthesizing textures on arbitrary surfaces tend to be slow. Such schemes typically produce high-quality results, and can be accelerated by using either tree-structured vector quantization or a “kd-tree” search for identifying candidate texture patches. However, such schemes typically perform poorly when using textures such as so-called “natural” textures. One scheme has been specially adapted to handle “natural” surface textures, however, this scheme does not generalize well to other types of textures.
Another problem with mapping textures to arbitrary surfaces is a lack of visual realism. Visual realism in digital images requires detailed attention to both surface geometry and surface details. With recent advances in surface texture synthesis, a real-world surface, such as a real surface that is digitally reconstructed from laser range scans, can be decorated with a texture that fits the surface naturally and seamlessly. However, conventional texturing of real world surfaces tends to lack realism because textures in traditional graphics represent only color or albedo variations on smooth surfaces. Real-world textures, on the other hand, arise from both spatially-variant surface reflectance and surface mesostructures, i.e., the small but visible local geometric details. Unfortunately, the mesostructures which are responsible for fine-scale shadows, occlusions, and specularities, are typically ignored by conventional textures mapping schemes.
Consequently, in an attempt to address this problem, one conventional scheme captures and renders the complex appearance of real world surfaces, by using polynomial texture maps (PTM's) for capturing the surface appearance under a fixed viewpoint but different lighting directions. For surface light fields, the appearance of surfaces under fixed light sources but different viewing directions are captured and stored in a compact way for rendering. In particular, PTM's are an extension to conventional texture maps that allow enhanced image quality. This enhanced image quality is accomplished by storing second-order bi-quadratic polynomial coefficients for each image pixel in addition to storing a color for each pixel. Consequently, these PTM's are capable of modeling changes to a pixel color based on either light source position or viewing direction.
Another approach for realistic texturing of surfaces uses a “bidirectional texture function” (BTF) to represent real-world textures for modeling surface mesostructures and reflectance variations. The BTF is a six-dimensional function whose variables include the two-dimensional pixel position, and the viewing and lighting directions. A BTF can be mapped onto surfaces using conventional texture mapping techniques such as those discussed above. However, BTF mapping on arbitrary surfaces can introduce inconsistent mesostructures. In particular, conventional schemes for texture mapping arbitrary surfaces typically use a collection of overlapping patches, with textures in the overlapping regions being blended to hide seams.
While this technique works well for many textures, for BTF's, blending often introduces inconsistent mesostructures, thereby reducing the realism of the texture mapped surface. Further, BTF mapping on arbitrary surfaces also suffers from other common problems of texture mapping, such as, for example, distortion, seams, and the need for considerable user intervention to create good-quality texture maps. Thus, the requirement of a consistent mesostructure for ensuring image realism is where surface BTF synthesis differs fundamentally from surface texture synthesis since conventional textures ignore mesostructures completely.
One conventional scheme considers mesostructures in synthesizing a continuous BTF for synthesizing images under different viewing and lighting settings. Unfortunately, this scheme is unable to efficiently accomplish surface BTF synthesis because it is too time consuming to reconstruct or render the appearance from the surface geometry for all lighting and viewing settings, especially for BTFs with complex geometry or radiance distributions. Further, this scheme has difficulty in handling real-world textures with complex bump structures or without dominant diffuse components.
Attempts to achieve a consistent mesostructure on a surface have been made by directly applying surface texture synthesis techniques to surface BTF synthesis. With such schemes, a sample BTF is regarded as a two-dimensional texture map, in which the BTF value at each pixel is a four-dimensional function of the viewing and lighting directions, with the four-dimensional function being discretized into a vector for texture synthesis. Unfortunately, this approach incurs a huge computational cost because of the large amount of data in a BTF sample. For example, at a resolution of 12×15×12×5, the BTF value at a pixel is a 10800-dimensional vector, as opposed to the usual three-dimensional RGB vectors for color pixels. Since texture synthesis time grows in direct proportion to the vector dimension, computing a surface BTF with such schemes can take days or even months. Principal Components Analysis (PCA) has been used to reduce the vector dimension; however, PCA has been unable to reduce the vector dimension significantly enough to alter the nature of the problem.
Another BTF-based texture synthesis scheme introduced three-dimensional “textons” based on an observation that, at a local scale, there are only a small number of perceptually distinguishable mesostructures and reflectance variations on a surface. Unfortunately, three-dimensional textons themselves are not compact because of their huge appearance vectors. Consequently, reconstructive BTF synthesis based on this scheme is not feasible on most surfaces because the basic operations for surface BTF synthesis, such as, for example, texton resampling, distance computation, and BTF reconstruction, are prohibitively computationally expensive with very large storage data requirements. For example, with this scheme, a surface BTF reconstruction requires that a four-dimensional function be assigned to each mesh vertex. Consequently, a surface mesh of only 250 K vertices will require approximately 2.5 Gbytes of storage.
Many conventional BTF synthesis schemes are designed for two-dimensional rectangles or rectangular surface patches, not for arbitrary surfaces. For example, “texture morphing” is a conventional technique for BTF synthesis which assumes that a surface is a simple height field with Lambertian reflectance. Another such scheme provides a BTF synthesis algorithm which is based on three-dimensional textons. This algorithm first synthesizes a two-dimensional map of texton labels using non-parametric sampling, and then reconstructs a BTF from the synthesized texton labels. However, one problem with this reconstructive synthesis of a BTF is the high computational costs for synthesizing texton labels and reconstructing the BTF using the huge appearance vectors of textons (e.g., an appearance vector is 57,600-dimensional if only 400 sample images of the BTF are used).
Therefore, what is needed is a system and method for efficiently synthesizing surface BTF-type textures which provide realistic texturing of arbitrary surfaces. Further, such a system and method should provide for compact data structures that minimize texture synthesis data storage requirements. Finally, such a system and method should rapidly synthesize realistic textures on arbitrary surfaces.